The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Recent studies have shown that CLIP has achieved remarkable success in performing zero-shot inference while its fine-tuning performance is not satisfactory. In this paper, we identify that fine-tuning performance is significantly impacted by hyper-parameter choices. We examine various key hyper-parameters and empirically evaluate their impact in fine-tuning CLIP for classification tasks through a comprehensive study. We find that the fine-tuning performance of CLIP is substantially underestimated. Equipped with hyper-parameter refinement, we demonstrate CLIP itself is better or at least competitive in fine-tuning compared with large-scale supervised pre-training approaches or latest works that use CLIP as prediction targets in Masked Image Modeling. Specifically, CLIP ViT-Base/16 and CLIP ViT-Large/14 can achieve 85.7%,88.0% finetuning Top-1 accuracy on the ImageNet-1K dataset . These observations challenge the conventional conclusion that CLIP is not suitable for fine-tuning, and motivate us to rethink recently proposed improvements based on CLIP. We will release our code publicly at \url{https://github.com/LightDXY/FT-CLIP}.

Copy-Paste is a simple and effective data augmentation strategy for instance segmentation. By randomly pasting object instances onto new background images, it creates new training data for free and significantly boosts the segmentation performance, especially for rare object categories. Although diverse, high-quality object instances used in Copy-Paste result in more performance gain, previous works utilize object instances either from human-annotated instance segmentation datasets or rendered from 3D object models, and both approaches are too expensive to scale up to obtain good diversity. In this paper, we revisit Copy-Paste at scale with the power of newly emerged zero-shot recognition models (e.g., CLIP) and text2image models (e.g., StableDiffusion). We demonstrate for the first time that using a text2image model to generate images or zero-shot recognition model to filter noisily crawled images for different object categories is a feasible way to make Copy-Paste truly scalable. To make such success happen, we design a data acquisition and processing framework, dubbed "X-Paste", upon which a systematic study is conducted. On the LVIS dataset, X-Paste provides impressive improvements over the strong baseline CenterNet2 with Swin-L as the backbone. Specifically, it archives +2.6 box AP and +2.1 mask AP gains on all classes and even more significant gains with +6.8 box AP +6.5 mask AP on long-tail classes.

Novel artificial intelligence (AI) technology has expedited various scientific research, e.g., cosmology, physics and bioinformatics, inevitably becoming a significant category of workload on high performance computing (HPC) systems. Existing AI benchmarks tend to customize well-recognized AI applications, so as to evaluate the AI performance of HPC systems under predefined problem size, in terms of datasets and AI models. Due to lack of scalability on the problem size, static AI benchmarks might be under competent to help understand the performance trend of evolving AI applications on HPC systems, in particular, the scientific AI applications on large-scale systems. In this paper, we propose a scalable evaluation methodology (SAIH) for analyzing the AI performance trend of HPC systems with scaling the problem sizes of customized AI applications. To enable scalability, SAIH builds a set of novel mechanisms for augmenting problem sizes. As the data and model constantly scale, we can investigate the trend and range of AI performance on HPC systems, and further diagnose system bottlenecks. To verify our methodology, we augment a cosmological AI application to evaluate a real HPC system equipped with GPUs as a case study of SAIH.

Federated learning (FL) enables the building of robust and generalizable AI models by leveraging diverse datasets from multiple collaborators without centralizing the data. We created NVIDIA FLARE as an open-source software development kit (SDK) to make it easier for data scientists to use FL in their research and real-world applications. The SDK includes solutions for state-of-the-art FL algorithms and federated machine learning approaches, which facilitate building workflows for distributed learning across enterprises and enable platform developers to create a secure, privacy-preserving offering for multiparty collaboration utilizing homomorphic encryption or differential privacy. The SDK is a lightweight, flexible, and scalable Python package, and allows researchers to bring their data science workflows implemented in any training libraries (PyTorch, TensorFlow, XGBoost, or even NumPy) and apply them in real-world FL settings. This paper introduces the key design principles of FLARE and illustrates some use cases (e.g., COVID analysis) with customizable FL workflows that implement different privacy-preserving algorithms. Code is available at https://github.com/NVIDIA/NVFlare.

尽管人工智能（AI）在理解各个领域的分子方面取得了重大进展，但现有模型通常从单个分子模态中获得单个认知能力。由于分子知识的层次结构是深刻的，即使人类也从不同的方式中学习，包括直觉图和专业文本，以帮助他们的理解。受到这一点的启发，我们提出了一个分子多模式基础模型，该模型是从分子图及其语义相关的文本数据（从发表的科学引用索引论文中爬立）的。该AI模型代表了直接桥接分子图和自然语言的关键尝试。重要的是，通过捕获两种方式的特定和互补信息，我们提出的模型可以更好地掌握分子专业知识。实验结果表明，我们的模型不仅在诸如跨模式检索和分子标题之类的跨模式任务中表现出有希望的性能，而且还可以增强分子属性预测，并具有从自然语言描述中产生有意义的分子图的能力。我们认为，我们的模型将对跨生物学，化学，材料，环境和医学等学科的AI能力领域产生广泛的影响。

本文提出了一个简单而有效的框架蒙版，该框架将新提出的掩盖自distillation纳入对比的语言图像预处理中。掩盖自distillation的核心思想是将表示从完整的图像提取到蒙版图像预测的表示形式。这种合并享有两个重要的好处。首先，掩盖的自我验证目标是本地贴片表示学习，这与视觉对比度的互补，专注于与文本相关的表示。二，掩盖的自我验证也与视觉语言对比符合训练目标的视野对比是一致的。视觉编码器用于功能对齐，因此能够学习本地语义从该语言中获得间接监督。我们提供了专门设计的实验，并进行了全面的分析，以验证这两个好处。从经验上讲，我们表明，当MaskClip应用于各种具有挑战性的下游任务时，可以在线性探测，填充和零拍摄中取得卓越的结果，并在语言编码器的指导下取得了卓越的结果。

多模式学习，尤其是大规模的多模式预训练，在过去的几年中已经迅速发展，并带来了人工智能（AI）的最大进步。尽管具有有效性，但了解多模式预训练模型的潜在机制仍然是一个巨大的挑战。揭示此类模型的解释性可能会使AI领域中新型学习范式的突破。为此，鉴于人脑的多模式性质，我们建议借助非侵入性脑成像技术（例如功能磁共振成像（fMRI））探索多模式学习模型的解释性。具体而言，我们首先提出了1500万个图像文本对预训练的新设计的多模式基础模型，该模型在各种认知下游任务中显示出强烈的多模式理解和概括能力。此外，从神经编码的角度来看（基于我们的基础模型），我们发现，与单峰相比，经过多模式训练的视觉和舌编码器都更像脑状。特别是，我们确定了许多大脑区域，其中多模式训练的编码器表现出更好的神经编码性能。这与现有有关探索大脑多感觉整合的研究的发现是一致的。因此，我们认为，多模式基础模型是神经科学家研究人脑中多模式信号处理机制的更合适的工具。我们的发现还证明了多模式基础模型作为理想的计算模拟器的潜力，以促进脑和大脑的AI研究。

尖峰神经网络是一种神经形态计算，据信可以提高智能水平，并为量子计算提供了前提。在这项工作中，我们通过设计一个光学尖峰神经网络来解决此问题，并证明它可以用于加速计算速度，尤其是在组合优化问题上。在这里，尖峰神经网络是由反对称耦合的退化光学参数振荡器脉冲和耗散脉冲构建的。选择非线性转移函数以减轻幅度不均匀性，并根据峰值神经元的动态行为破坏所得的局部最小值。从数值上证明，尖峰神经网络协会机器在组合优化问题上具有出色的性能，预计将为神经计算和光学计算提供新的应用程序。

土地覆盖分类是一项多级分割任务，将每个像素分类为地球表面的某些天然或人为类别，例如水，土壤，自然植被，农作物和人类基础设施。受硬件计算资源和内存能力的限制，大多数现有研究通过将它们放置或将其裁剪成小于512*512像素的小斑块来预处理原始遥感图像，然后再将它们发送到深神经网络。然而，下调图像会导致空间细节损失，使小细分市场难以区分，并逆转了数十年来努力获得的空间分辨率进度。将图像裁剪成小斑块会导致远程上下文信息的丢失，并将预测的结果恢复为原始大小会带来额外的延迟。为了响应上述弱点，我们提出了称为Mkanet的有效的轻巧的语义分割网络。 Mkanet针对顶视图高分辨率遥感图像的特征，利用共享内核同时且同样处理不一致的尺度的地面段，还采用平行且浅层的体系结构来提高推理速度和友好的支持速度和友好的支持图像贴片，超过10倍。为了增强边界和小段歧视，我们还提出了一种捕获类别杂质区域的方法，利用边界信息并对边界和小部分错误判断施加额外的惩罚。广泛实验的视觉解释和定量指标都表明，Mkanet在两个土地覆盖分类数据集上获得了最先进的准确性，并且比其他竞争性轻量级网络快2倍。所有这些优点突出了Mkanet在实际应用中的潜力。